Mending the Methodological Mistakes

Preparation of this chapter was supported by National Science Foundation Grant no. 1024293 and a grant from the Brady Education Foundation awarded to ASL, and a National Science Foundation Graduate Research Fellowship awarded to EDS. Play is too often in the American crosshairs, with those who claim it is a waste of time on the one hand and those who claim it is crucially important to development on the other. Putting the mind and body in a relaxed state and letting the imagination flow to new and different possibilities seems to us like it should be a net positive. But then why doesn’t the research show this more definitively? We recently reviewed dozens of studies of the impact of pretend play on many aspects of children’s development and concluded that there is no clear evidence that the activity helps any aspect (Lillard et al., 2013). But the quality of the studies is problematic. It might be that those who study play are often already so convinced of its benefits at the outset—what Peter K. Smith (1988) has called “the play ethos”—that they apply less rigorous research standards than they might in other domains. Here we review several “methodological mistakes” that must be remedied to bring about a better database from which to understand the role of play in development. The problems are discussed in the order in which they appear in experiment execution and analysis. They are (1) nonrandom assignment to groups, (2) knowledgeable experimenters, (3) implementer confounded with implementation, (4) teaching to the test and otherwise nonparallel control conditions, (5) unrigorous data analytic practices, and (6) skewed interpretation of results.